Information processing apparatus, control method thereof, and   storage medium

ABSTRACT

To make it possible to easily extract additional information from printed matter by using a built-in camera of a mobile terminal. An information processing apparatus that reads additional information by capturing printed matter in which the additional information is embedded by using a built-in camera, including: a control unit configured to acquire a first captured image obtained by capturing the printed matter under a first image capturing condition and a second captured image obtained by capturing the printed matter under a second image capturing condition different from the first image capturing condition; and an extraction unit configured to extract the additional information by analyzing the first captured image and the second captured image.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a reading technique of informationembedded in printed matter.

Description of the Related Art

An information hiding technique is known that adds digital informationindicating an author, whether or not use thereof is permitted, and so onto printed matter obtained by printing a photo, a painting, and so on bymultiplexing the digital information so as to be difficult to identifyvisually. It is possible to take out digital information added toprinted matter (hereinafter, called “additional information”) bycapturing the target printed matter by an image capturing device, suchas a scanner and a built-in camera of a mobile terminal, and analyzingthe captured image. Then, as a technique to appropriately take outadditional information, for example, Japanese Patent Laid-Open No.2005-026797 has disclosed a technique to attach a mark (correctionmarker) indicating a reference of an image at the time of generating awatermark-embedded image. In the case of this method, by estimating adegree in which a reference point or a reference line is distorted froma predetermined state by using a correction marker to correct an imageat the time of taking out additional information, resistance of anelectronic watermark for complex geometric transformation, such asprojection transformation, is implemented.

In order to correctly take out additional information embedded by theinformation hiding technique from printed matter, it is generallyimportant to obtain an image of high quality with as less blurring ordistortion as possible by performing image capturing at the correctposition with respect to the printed matter. However, it is originallydifficult to perform image capturing under a stable condition in thecase where image capturing is performed by a built-in camera of a mobileterminal, such as a smartphone. For example, the distance from theprinted matter is too long or rotation or trapezoidal distortion occurs.In this respect, the above-described method that uses a correctionmarker premises that image capturing is performed correctly except forgeometric distortion. However, in handheld image capturing using abuilt-in camera of a mobile terminal, camera shake occurs or shadow orlight is apt to be cast on the printed matter. Further, even in the casewhere image capturing is performed in the state where there is noproblem other than distortion, on a condition that inclination of amobile terminal (built-in camera) with respect to the printed matterincreases, for example, such a case occurs where only a part of an areain which additional information is embedded is in focus. In such a case,a satisfactory image is not obtained even by performing geometrictransformation and it is not possible to correctly extract additionalinformation. Further, in the case where the amount of additionalinformation increases, the basic unit of the area in which additionalinformation is embedded becomes large and image capturing of an imageenabling accurate extraction of additional information becomes moredifficult. As described above, in the case where additional informationis read from printed matter by using the camera function of a mobileterminal, there is a possibility that obtaining a captured imagesuitable to extraction of additional information becomes complicated.

Consequently, an object of the present invention is to make it possibleto easily extract additional information from printed matter by using abuilt-in camera of a mobile terminal.

SUMMARY OF THE INVENTION

The information processing apparatus according to the present inventionis an information processing apparatus that reads additional informationby capturing printed matter in which the additional information isembedded by using a built-in camera, including: a control unitconfigured to acquire a first captured image obtained by capturing theprinted matter under a first image capturing condition and a secondcaptured image obtained by capturing the printed matter under a secondimage capturing condition different from the first image capturingcondition; and an extraction unit configured to extract the additionalinformation by analyzing the first captured image and the secondcaptured image.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A and FIG. 1B are block diagrams showing main functions in animage forming apparatus and a mobile terminal making up an imageprocessing system;

FIG. 2A and FIG. 2B are block diagrams showing hardware configurationsof the image forming apparatus and the mobile terminal;

FIG. 3 is a diagram showing an internal configuration of an embedmentprocessing unit;

FIG. 4 is a flowchart showing a flow of processing to embed additionalinformation in image information;

FIG. 5 is an explanatory diagram of an embedment area;

FIG. 6A to FIG. 6D are explanatory diagrams of a quantization condition;

FIG. 7 is a block diagram showing an internal configuration of anextraction processing unit;

FIG. 8A and FIG. 8B are diagrams showing coefficients in spatialfilters;

FIG. 9 is a diagram showing a two-dimensional frequency area;

FIG. 10A and FIG. 10B are diagrams showing an example of a unit block;

FIG. 11A and FIG. 11B are diagrams showing the way additionalinformation represented by a unit block is embedded repeatedly;

FIG. 12A and FIG. 12B are diagrams showing the way additionalinformation corresponding to one set is read from a plurality of unitblocks;

FIG. 13 is a diagram showing the way handheld image capturing isperformed for printed matter in the state where a mobile terminal isinclined;

FIG. 14 is a diagram showing the way printed matter is viewed through alens of a mobile terminal at the time of handheld image capturing;

FIG. 15 is a flowchart showing a flow of image capturing control at thetime of extracting additional information from printed matter; and

FIG. 16 is a flowchart showing a flow of image capturing control at thetime of extracting additional information from printed matter.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, with reference to the attached drawings, the presentinvention is explained in detail in accordance with preferredembodiments. Configurations shown in the following embodiments aremerely exemplary and the present invention is not limited to theconfigurations shown schematically.

First Embodiment

FIG. 1A and FIG. 1B are block diagrams showing main functions in animage forming apparatus 100 and a mobile terminal 110 making up an imageprocessing system according to the present embodiment. The image formingapparatus 100 shown in FIG. 1A is a printing apparatus, for example,such as an MFP (Multi Function Peripheral), having a print function. Themobile terminal 110 shown in FIG. 1B is a mobile information processingapparatus, for example, such as a smartphone and a tablet PC, having acamera function. In the present embodiment, the image forming apparatus100 outputs printed matter in which additional information is embeddedby multiplexing. The additional information is information, for example,such as voices, moving images, still images, and characters, relating toa printed image, or management information, such as the copyright, thedate and time of image capturing, the image capturing place, and theimage capturer, relating to the image. The additional information may beincluded as part of image information made up as an image file. Then,the additional information is taken out by capturing the printed matterby a built-in camera (image capturing sensor) 111 of the mobile terminal110 and analyzing the obtained captured image. There is a case whereadditional information is called “multiplexed information” or “embeddedinformation”.

The image forming apparatus 100 includes an embedment processing unit101 configured to perform embedment processing by multiplexingadditional information in image information and a print unit (printerengine) 102 configured to perform printing processing by using imageinformation in which additional information is embedded. The print unit102 is a printer, for example, such as an ink jet printer and a laserprinter, which implements gradation representation on a printing medium,such as paper, by using pseudo gradation processing. The embedmentprocessing unit 101 is implemented as a printer driver that generatesimage information to be delivered to the print unit 102, or asapplication software. Further, the embedment processing unit 101 mayalso be implemented in an aspect in which it is incorporated in a copymachine, a facsimile, a printer main body, and so on as hardware orsoftware. There is a case where the processing to embed additionalinformation is called “multiplexing processing”.

The mobile terminal 110 has an extraction processing unit 112 configuredto perform processing to extract additional information from printedmatter and an additional information check unit 113 configured toperform error detection of extracted additional information, in additionto the above-described image capturing sensor 111. The extractionprocessing unit 112 and the additional information check unit 113 may beimplemented by software or hardware. There is a case where theprocessing to take out embedded additional information is called“reading processing” or “separation processing”.

In the present embodiment, a configuration in which the embedmentprocessing unit 101 and the print unit 102 are included in the imageforming apparatus 100 is explained as an example, but these may beconfigured as separate apparatuses. For example, the configuration maybe one in which an information processing apparatus, such as a PC, hasthe function of the embedment processing unit 101 and a printingapparatus independent of the information processing apparatus has thefunction of the print unit 102.

[Basic Configuration of Image Forming Apparatus]

The embedment processing unit 101 embeds (multiplexes) additionalinformation in image information in the state where it is difficult tovisually identify the additional information. The print unit 102performs printing processing based on the image information in which theadditional information is embedded created by the embedment processingunit 101.

FIG. 2A is a diagram showing an example of a hardware configuration ofthe image forming apparatus 100. A CPU 200 is a processor thatcentralizedly controls the image forming apparatus 100. A ROM 201 and anHDD 203 store basic programs and control programs of the image formingapparatus 100, various applications, data, and so on. The CPU 200implements various functions, such as the embedment processing unit 101described previously, by reading programs stored in the ROM 201 onto aRAM 202 and executing the programs. The RAM 202 is used also as a workmemory of the CPU 200.

A network interface 204 has a configuration in accordance with theaspect of a network, for example, such as a wired network and a wirelessnetwork. Further, it is also possible to communicate with the mobileterminal 110 by a plurality of kinds of wireless network whosecommunication ranges are different, for example, such as near fieldcommunication (NFC) whose communication distance is several centimeters.

A display 205 displays each setting screen, a preview screen, and so onto a user. An operation unit 206 has, for example, a keyboard, a touchpanel, and so on and is capable of receiving operation instructions froma user. A device I/F 207 connects the print unit 102 (printer engine)and a system bus 209. In FIG. 1A, the print unit 102 is shown, but inaccordance with the function that the image forming apparatus 100 has, aprocessing unit, such as a scanner and a FAX, is connected to the deviceI/F 207. An image processing unit 208 performs image processing inaccordance with uses for image data acquired from the outside. Forexample, the image processing unit 208 performs processing, such ascolor space conversion, binarization processing, and enlargement,reduction, and rotation of an image, in accordance with the printingmethod of the print unit 102.

[Basic Configuration of Mobile Terminal]

In the case where a user performs image capturing of printed matter byusing an image reading application installed in the mobile terminal 110,additional information is extracted from a captured image in theextraction processing unit 112 and error detection is performed in theadditional information check unit 113.

FIG. 2B is a diagram showing an example of a hardware configuration ofthe mobile terminal 110. The mobile terminal 110 includes aconfiguration of a general-purpose information processing apparatus. ACPU 210 is a processor that centralizedly controls the mobile terminal110. The CPU 210 implements various functions including the extractionprocessing unit 112 and the additional information check unit 113described previously by reading programs stored in a ROM 211 onto a RAM212 and executing the programs. The ROM 211 stores basic programs andcontrol programs of the mobile terminal 110, various applications, data,and so on. The RAM 212 is used also as a work memory of the CPU 210.Flowcharts in FIG. 15 and FIG. 16, to be described later, areimplemented by the CPU 210 reading a program corresponding to each stepand executing the program. In the case where additional informationalready checked by the additional information check unit 113 is, forexample, image information or character information, the contentsthereof are output to a display 215 and in the case where the additionalinformation is link information, such as URL, a browser is activated andthen a link destination is displayed on the display 215. As arepresentative cause of an error in the additional information checkunit 113, mention is made of a problem of poor image quality, such ascamera shake and out of focus, resulting from handheld image capturingby a mobile terminal. The image capturing sensor 111 performs imagecapturing of a still image by using AF (Auto Focus), AE (Auto Exposure),and AWB (Auto White Balance). In the case where the captured still imagehas a problem and an error is detected in the extracted additionalinformation, the operation to perform image capturing again by changingthe condition is repeated until the check is passed. By doing so, eventhough the attitude at the time of image capturing is unstable, it ismade possible to extract additional information. It may also be possibleto cut out a still image from an obtained moving image by performingimage capturing in the moving image mode. Further, it may also bepossible for a user to perform image capturing while manually changingat least one of the image capturing conditions, such as focus position,exposure, shutter speed, and white balance, without making use of AF,AE, and AWB described above. Furthermore, it may also be possible tochange the image capturing condition by another method. For example, itmay also be possible for at least one condition to be changed manuallyby a user, for at least one condition to be fixed, and for at least onecondition to be automatically determined by an image readingapplication.

A network I/F 213 has a configuration in accordance with the aspect of anetwork, for example, such as a wired network and a wireless network.Further, it is also possible to communicate with the image formingapparatus 100 by a plurality of kinds of wireless network whosecommunication ranges are different, for example, such as near fieldcommunication (NFC) whose communication distance is several centimeters.The display 215 displays each setting screen, a preview screen, and soon to a user. An operation unit 214 has, for example, a hard key and thelike and is capable of receiving operation instructions from a user.Further, the operation unit 214 and the display 215 may be integratedinto one unit, for example, a so-called touch panel. A distance sensor216 is a sensor that measures the distance to an object, such as printedmater. Each block shown in FIG. 2B is connected so as to be capable ofcommunication with one another via a system bus 217.

[Embedment Processing of Additional Information]

First, how additional information is embedded in image information isexplained. FIG. 3 is a diagram showing an internal configuration of theembedment processing unit 101. The embedment processing unit 101includes an error diffusion unit 301, a block generation unit 302, and aquantization condition control unit 303.

The error diffusion unit 301 performs quantization processing using theerror diffusion method for input image information. Due to this, imageinformation is generated, which represents gradation properties in termsof area by pixel values whose gradation number is smaller than the inputgradation number. In the following, explanation is given by taking thecase of binary quantization as an example. The block generation unit 302performs processing to divide input image information into predeterminedunit areas (block generation). The unit area is, for example, arectangular area of 96×96 pixels and hereinafter called a “unit block”.The shape of the unit block is not limited to a rectangle and a shapeother than a rectangle may be accepted. The quantization conditioncontrol unit 303 sets a quantization condition for each unit blockobtained by the block generation unit 302 based on additionalinformation input separately from (or along with) the image information.

Next, a flow of processing to embed additional information in imageinformation is explained with reference to the flowchart in FIG. 4. Theseries of processing shown in FIG. 4 is implemented by the CPU 200 ofthe image forming apparatus 100 reading a program stored in the ROM 201onto the RAM 202 and executing the program.

At step 401, a variable i secured in the RAM 202 is initialized. Here,the variable i is a variable that counts the address in the verticaldirection. At step 402, a variable j secured in the RAM 202 isinitialized. Here, the variable j is a variable that counts the addressin the horizontal direction. Following the above, at step 403, whetheror not the pixel (pixel of interest) corresponding to coordinates (i,j), which are the current processing address, belongs to the embedmentprocessing-target area (embedment area) is determined.

Here, with reference to FIG. 5, the embedment area is explained. FIG. 5shows one image 500 representing the number of horizontal pixels byWIDTH and the number of vertical pixels by HEIGHT. By taking thetop-left point of the image 500 as the origin, block generation isperformed with a block whose number of horizontal pixels is N and whosenumber of vertical pixels is M. Here, the block generation is performedwith the origin as a reference point, but a point separate from theorigin may be used as a reference point. In order to embed informationin this image 500 as much as possible, multiplexed blocks of N×M pixelsare arranged in order from the reference point. That is, in the casewhere the number of blocks that can be arranged in the horizontaldirection is taken to be W and the number of blocks that can be arrangedin the vertical direction is taken to be H, W and H are expressed byexpression (1) and expression (2) below.

W=INT(WIDTH/N)   expression (1)

H=INT(HEIGHT/M)   expression (2)

Here, INT( ) indicates the integer part of a value within ( ).

The number of pixels, which is the reminder of division in expressions(1) and (2) described above, correspond to the end portion at the timeof arranging a plurality of unit blocks of N×M pixels and this portionis outside the embedment area. Explanation is returned to the flow inFIG. 4.

Then, in the case where it is determined that the pixel of interest isnot inside the embedment area (outside the embedment area), theprocessing advances to step 404 and a quantization condition C is set.On the other hand, in the case where it is determined that the pixel ofinterest is inside the embedment area, the processing advances to step405 and additional information to be multiplexed is read. Here, forconvenience of explanation, additional information is represented by anarray, such as code [ ]. For example, in the case where additionalinformation is supposed to be information corresponding to 48 bits, inthe array code [ ], one bit is stored in each of code [0] to code [47].Then, by substituting information within the array code [ ] expressed byexpression (3) below in a variable bit secured in the RAM 202,additional information is read.

bit=code [INT(i/M)×W+INT(j/N)]  expression (3)

At step 406 that follows, whether or not the substituted variable bit is“1” is determined. As described previously, the information within thearray code [ ] is stored one bit each, and therefore, the value of thevariable bit also indicates one of “0” and “1”. Here, in the case where“0” is determined, a quantization condition A is set at step 407. On theother hand, in the case where “1” is determined, a quantizationcondition B is set at step 408.

Here, the quantization conditions A, B, and C are explained. Thequantization conditions in the error diffusion method include variousfactors and here, a quantization threshold value. The quantizationcondition C that is set at step 404 is used in the case where the pixelof interest is outside the embedment area, and therefore, thequantization threshold value may be any condition. As describedpreviously, in the case where the quantization levels are two values ingradation representation in which one pixel is eight bits, “255”, whichis the maximum value, and “0”, which is the minimum value, arequantization representative values and “128”, which is the intermediatevalue, is set as the quantization threshold value in many cases. Thequantization condition C in the present embodiment is also set to acondition whose quantization threshold value is taken to be a fixedvalue of “128”. The quantization condition A that is set at step 407 andthe quantization condition B that is set at step 408 are used in thecase where the pixel of interest is inside the embedment area and theunit block is made up, and therefore, it is necessary to make adifference in image quality by different quantization conditions.However, it is necessary for the difference in image quality to berepresented so as not to be visually identified easily but to be capableof being identified easily from the paper surface. FIG. 6A and FIG. 6Bare explanatory diagrams of the quantization conditions A and B. FIG. 6Ais a diagram showing a period of a change in the quantization thresholdvalue in the quantization condition A. In FIG. 6A, it is assumed thatone cell corresponds to one pixel and a white cell is a fixed thresholdvalue and a black cell is a fluctuating threshold value. That is, in theexample in FIG. 6A, a matrix of eight horizontal pixels and fourvertical pixels is formed and only for black cells, a largely shiftedvalue is set as a threshold value. FIG. 6B is a diagram showing a periodof a change in the quantization threshold value in the quantizationcondition B. In the example in FIG. 6B, different from FIG. 6A, a matrixof four horizontal pixels and eight vertical pixels is formed andsimilarly, only for black cells, a largely shifted value is set as athreshold value. Then, in the case where one pixel is an 8-bit tonelevel value, for example, as a fixed threshold value, “128” is set andas a largely shifted value, “10” is set. In the case where thequantization threshold value becomes small, the quantized value of thepixel of interest easily becomes “1”. That is, in the arrangement of theblack cells both in FIG. 6A and in FIG. 6B, the quantized value “1”becomes more likely to be put side by side. In other words, the unitblock of N×M pixels in which dots occur in the arrangement of blackcells shown in FIG. 6A and the unit block of N×M pixels in which dotsoccur in the arrangement of black cells shown in FIG. 6B exist mixedly.Some change in the quantization threshold value in the dither diffusionmethod does not affect significantly in terms of image quality. In theordered dither method, depending on the dither pattern to be used, theimage quality of gradation representation is affected significantly.However, in the error diffusion method regularly given a change in thequantization threshold value, the gradation representation determiningthe image quality is the error diffusion method, and therefore, even inthe case where the arrangement of dots changes somewhat or theoccurrence of texture changes, the image quality of gradationrepresentation is hardly affected. Even in the case where thequantization threshold value changes, the error that causes a differencebetween the signal value and the quantized value is diffused to theperipheral pixels, and therefore, the input signal value is saved in amacro manner. That is, concerning the arrangement of dots and theoccurrence of texture in the error diffusion method, it can be said thatredundancy is extremely large. It may also be possible to performembedment of additional information by taking FIG. 6C in which a patternof 8×8 pixels combining the pattern in FIG. 6A is taken to be theminimum unit and FIG. 6D in which a pattern of 8×8 pixels combining thepattern in FIG. 6B is taken to be the minimum unit as the quantizationconditions A and B, respectively. Explanation is returned to the flow inFIG. 4.

At step 409, based on the quantization conditions set as describedabove, quantization processing by the error diffusion method isperformed. Then, at step 410, the variable j is updated (incremented)and at step 411, whether or not the variable j after updating is lessthan the number of horizontal pixels WIDTH is determined. In the casewhere the variable j is determined to be less than WIDTH, the processingreturns to step 403, and the pixel located on the right side isdetermined to be the pixel of interest and the processing is repeated.On the other hand, in the case where the variable j is determined not tobe less than WIDTH (in the case where the processing in the horizontaldirection is completed for the number of pixels corresponding to WIDTH),the processing advances to step 412 and the variable i is updated(incremented). Then, at step 413, whether or not the variable i afterupdating is less than the number of vertical pixels HEIGHT isdetermined. In the case where the variable i is determined to be lessthan HEIGHT, the processing returns to step 402, and the pixel locatedbelow is determined to be the pixel of interest and the processing isrepeated. On the other hand, in the case where the variable i isdetermined not to be less than HEIGHT (in the case where the processingin the vertical direction is completed for the number of pixelscorresponding to HEIGHT), this processing is terminated.

The above is the contents of the embedment processing of additionalinformation. In this manner, the embedment processing of additionalinformation is performed in accordance with the quantization conditionset for each unit block. As described above, in the present embodiment,multiplexing is implemented by superposing predetermined periodicityindicating a sign on the quantization threshold value of the errordiffusion method. However, it may also be possible to implementmultiplexing by another superposition method. For example, it may alsobe possible to implement multiplexing by a method of directlysuperposing periodicity on the RGB values (luminance information).Alternatively, it may also be possible to implement multiplexing by amethod of superposing periodicity by separating RGB values into anothercolor space information (for example, CIE L*a*b, YCrCb signal), such asluminance-color difference information. Alternatively, it may also bepossible to implement multiplexing by a method of superposingperiodicity by separating RGB values into ink colors (for example, CMYKsignal).

[Extraction Processing of Additional Information]

Next, extraction of additional information is explained. In the presentembodiment, additional information is extracted by making use of aplurality of images captured by the mobile terminal 110. FIG. 7 is ablock diagram showing an internal configuration of the extractionprocessing unit 112. Explanation is given on the assumption thatadditional information is extracted from printed matter in which theadditional information is embedded by the embedment processing unit 101described previously. As a matter of course, the amount of additionalinformation per unit block at the time of embedment in the embedmentprocessing unit 101 and the amount of extracted information per unitblock at the time of extraction in the extraction processing unit 112are equal.

Image information obtained by capturing printed matter by the mobileterminal 110 is first input to a block generation unit 701. Here, it ispreferable for the resolution (image capturing resolution) of the imagecapturing sensor 111 of the mobile terminal 110 to be higher than orequal to the printing resolution at the time of creating printed matter.In order to accurately read scattered dot information on the printedmatter, it is necessary for the image capturing sensor 111 side to havea resolution higher than or equal to twice that on the printingapparatus side by the sampling theorem. However, with a resolutionhigher than or equal thereto, it is possible to determine scattereddots.

The block generation unit 701 divides an image into unit blocks made upof a number of horizontal pixels P and a number of vertical pixels Q.The size of this unit block is smaller than or equal to that of the unitblock at the time of embedment. That is, between the unit block at thetime of embedment and the unit block at the time of extraction, arelationship of P≤N and Q≤M holds. In the following, explanation isgiven on the assumption that the size of the unit block is the same atthe time of embedment and at the time of extraction (P=N and Q=M).

As spatial filters, a first spatial filter 702 and a second spatialfilter 703 whose characteristics are different from each other exit. Afiltering unit 704 performs digital filtering to calculate a product anda sum with a peripheral pixel. The coefficients in the first spatialfilter 702 and the second spatial filter 703 are set in accordance withthe period of a fluctuation threshold value of the quantizationcondition. Here, it is assumed that the quantization conditions are setby using the two kinds of periodicity shown in FIG. 6A and FIG. 6B andadditional information is embedded. The coefficients in the firstspatial filter 702 and the second spatial filter 703 used in theextraction processing unit 112 in this case are, for example, those asshown in FIG. 8A and FIG. 8B, respectively. In the spatial filters of5×5 pixels shown in FIG. 8A and FIG. 8B, the pixel located at the centerthereof is the pixel of interest and the other 24 pixels are peripheralpixels. In FIG. 8A and FIG. 8B, the blank portion where there is novalue indicates the pixel whose filer coefficient is “0”. As is obviousfrom each filter coefficient shown in FIG. 8A and FIG. 8B, each spatialfilter is an edge-enhancement filter. Then, the directionality of theedge to be enhanced and the directionality of the fluctuation thresholdvalue at the time of multiplexing coincide with each other in thecombination of FIG. 8A and FIG. 6A and in the combination of FIG. 8B andFIG. 6B, respectively. That is, the spatial filters are created so thatFIG. 8A coincides with FIG. 6A and FIG. 8B coincides with FIG. 6B.

As thinning units, a first thinning unit 705 corresponding to the firstspatial filter 702 and a second thinning unit 706 corresponding to thesecond spatial filter 703 exist. Then, in each thinning unit, thinningprocessing is performed by separating the signal (hereinafter,“converted value”) after filtering within the unit block into theregularity of periodicity and the regularity of phase, respectively.That is, between the first thinning unit 705 and the second thinningunit 706, the periodicity of thinning is different from each other andin each unit, a plurality of pieces of thinning processing is performedby changing the phase.

A converted value addition unit 707 adds the converted values of thepixels thinned by the first and second thinning units 705 and 706,respectively, for each phase. The thinning processing and the additionprocessing of the converted value correspond to extracting power of apredetermined frequency vector enhanced by the spatial filter.

A variance calculation unit 708 calculates a variance of a plurality ofadded values added for each phase in each periodicity. A determinationunit 709 determines the embedded sign based on the variance calculatedin each periodicity. FIG. 9 is a diagram showing a two-dimensionalfrequency area. The horizontal axis represents the frequency in thehorizontal direction and the vertical axis represents the frequency inthe vertical direction. The origin, which is the center, represents thedirect-current component and as becoming more distant from the origin,the area becomes a higher-frequency area. In FIG. 9, a circle 900 whosecenter is the intersection of the horizontal axis and the vertical axisrepresents a cut-off frequency by error diffusion. The filtercharacteristics in the error diffusion method indicate thecharacteristics of a high-pass filter whose low-frequency area is cutoff and the frequency to be cut off changes in accordance with thedensity of a target image. In the present embodiment, the frequencycharacteristics that occur after quantization change in accordance withthe set quantization threshold value and in the example of thequantization threshold value in FIG. 6A, a large power spectrum occurson the frequency vector indicated by a straight line 901 in FIG. 9.Further, in the example of the quantization threshold value in FIG. 6B,a large power spectrum occurs on the frequency vector indicated by astraight line 902 in FIG. 9. At the time of extraction of additionalinformation, by detecting the frequency vector on which this large powerspectrum occurs, the determination of the embedded sign is performed. Inthe present embodiment, it is assumed that each frequency vector isindividually enhanced and extracted. From the above, it can be said thatthe spatial filters shown in FIG. 8A and FIG. 8B described previouslyare high-pass filters having the directionality of a specific frequencyvector. That is, with the spatial filter in FIG. 8A, it is made possibleto enhance the frequency vector on the straight line 901 in FIG. 9 andwith the spatial filter in FIG. 8B, it is made possible to enhance thefrequency vector on the straight line 902 in FIG. 9. For example, it isassumed that by the setting of the quantization threshold value shown inFIG. 6A, a large power spectrum has occurred on the frequency vector ofthe straight line 901 in FIG. 9. At this time, with the spatial filterin FIG. 8A, the amount of change in power spectrum is amplified, butwith the spatial filter in FIG. 8B, the amount of change is hardlyamplified. That is, in the case where filtering is performed in parallelfor a plurality of spatial filters, amplification occurs only in thespatial space where the frequency vectors coincide and there is almostno amplification in the spatial filters other than that. Consequently,it is possible to easily determine on which frequency vector a largepower spectrum has occurred.

The additional information extracted by the extraction processing unit112 is checked for its accuracy by the additional information check unit113 and in the case where there is an error, the reading is determinedto be an error. As a check method, it may be possible to use a publiclyknown technique, such as CRC (Cyclic Redundancy Check), but such atechnique is not the feature of the present invention, and therefore,detailed explanation thereof is omitted here.

[Review of Problem of the Present Invention]

Here, the problem of the present invention is reviewed. FIG. 10A is adiagram showing a unit block in the case where additional information isembedded under the quantization conditions shown in FIG. 6C and FIG. 6Ddescribed previously. A thick-line rectangle 1000 represents one unitblock and by reading all the pieces of information within the unit block1000, it is possible to extract embedded additional information. Theunit block 1000 is made up of an anchor portion indicated by a whitearea 1001 and a data portion indicated by a dot area 1002. The anchorportion 1001 is an area having a predetermined size (for example, 32×32pixels), which is made use of to specify the origin of the unit block1000 at the time of reading of additional information. Then, to theanchor portion 1001, information is not added (blank) or arbitraryinformation indicating that the area is an anchor is embedded. The dataportion 1002 is covered over with patterns of 8×8 pixels 1003 shown inFIG. 6C or FIG. 6D. Here, it is assumed that the size of the unit block1000 is 96×96 pixels and the data portion 1002 is covered over with the128 patterns 1003. In this case, one pattern 1003 can represent 1-bitinformation, and therefore, there is an amount of 128-bit information inthe entire unit block 1000. In the present embodiment, the 128 bits arefurther separated into eight groups, each group having a total of 16bits of “DATA 12 bit” and “CRC 4 bit”, as shown in FIG. 10B. In thiscase, the actual capacity of data is 12 bits×8 groups=96 bits.

FIG. 11A is a diagram showing the way additional information representedby the unit block 1000 shown in FIG. 10A is embedded repeatedly withinprinted matter. That is, additional information is multiplexed on imageinformation on the printed matter and embedded repeatedly in units ofblocks. In order to read the additional information embedded in theprinted matter, it is only required to be capable of reading one unitblock of a plurality of unit blocks embedded repeatedly and it is notnecessarily required to read all the unit blocks. For example, as shownin FIG. 12A, there is a case where a shadow is cast at the time of imagecapturing by the mobile terminal 110. In this case, it is not possibleto cover all of the above-described eight groups by one unit blockalone, and therefore, there is a case where only several groups can beread. Even in such a case, by compensating for information of theportions that cannot be read because of the cast shadow by a group ofother unit blocks, it is made possible to read the additionalinformation corresponding to one unit block. That is, as shown in FIG.12B, by integrating three data portions corresponding to an area 1201with diagonal lines extending downward to the right, an area 1202 withdiagonal lines extending downward to the left, and an area 1203 with adiagonal grid, each being read from different unit blocks, all the dataportions corresponding to one unit blocks are obtained. Due to this,even in the case where the entire unit block is not captured sharply, itis possible to read additional information corresponding to the unitblock. However, there may be a case where unit blocks capable of mutualcomplementation do not exist in a captured image obtained by one-timeimage capturing. For example, in the case where two adjacent unit blocksmake up a set corresponding to one piece of additional information asshown in FIG. 11B, the possibility becomes strong that all unit blockscapable of mutual complementation do not exist in a captured imageobtained by one-time image capturing. Then, in the case where the numberof unit blocks making up one set becomes three or more, it becomes moredifficult to completely extract additional information as the numberincreases.

As described above, in the case of an image obtained by held imagecapturing using the mobile terminal 110, there may occur a case whereadditional information cannot be taken out accurately even by linkingdata portions partially read from a plurality of unit blocks. With suchcircumstances in mind, in the present invention, a plurality of capturedimages whose image capturing conditions are different is used forextraction of additional information. In the following, detailedexplanation is given.

[Acquisition of Captured Image Suitable to Extraction of AdditionalInformation]

FIG. 13 shows the way handheld image capturing is performed in the statewhere the mobile terminal 110 is inclined with respect to the printedmatter shown in FIG. 11A and FIG. 11B. Further, FIG. 14 shows views ofthe printed matter through a lens 1300 of the mobile terminal 110 at thetime of image capturing. As is obvious from FIG. 14, the shape of theprinted matter changes into a trapezoid with the side located on thefront being the long side and it is known that a distortion occurs.Then, the distances between each of positions α, β, and γ (see FIG. 13)on the printed matter and the lens 1300 of the mobile terminal 110 arequite different. Here, it is assumed that an image without blurring in arange indicated by a two-dot chain line 1402 in FIG. 14 is obtained inthe case where image capturing is performed by bringing the position αinto focus. In the case where a large shadow is captured in the capturedimage thus obtained, extraction of additional information will fail.Consequently, in the case where extraction of additional informationfails, image capturing is performed by changing the image capturingcondition. For example, an image without blurring in a range indicatedby a one-dot chain line 1401 is captured by bringing the position β onthe printed matter into focus, an image without blurring in a rangeindicated by a broken line 1403 is captured by bringing the position γinto focus, and so on.

FIG. 15 is a flowchart showing a flow of image capturing control foracquiring a captured image suitable to extraction of additionalinformation. In the present embodiment, automatic execution of theseries of processing by an image reading application (program) installedin the mobile terminal 110 is supposed, but it may also be possible tomanually perform partial processing, such as changing of the imagecapturing condition. In the following, detailed explanation is givenalong the flow in FIG. 15.

At step 1501, a captured image of printed matter is acquired in responseto image capturing instructions of a user via the operation unit 214 ofthe mobile terminal 110. The image capturing condition at this time isarbitrary and it may be possible to perform image capturing by using AF,AE, AWB, and so on described previously. Data of the acquired capturedimage is stored in an area secured within the RAM 212 after geometrictransformation, such as affine transformation, is performed therefor asneeded.

At step 1502 that follows, the anchor described previously is searchedfor by taking the acquired captured image as a target. In the presentembodiment, the portion of the anchor is covered over with patternswhose read value is “0” and in the case where “0” continues for 32pixels in the horizontal direction, the row is determined to be an“anchor row”. Then, in the case where the “anchor row” continues for 32pixels in the vertical direction, the area is determined to be ananchor. By performing this processing for the entire captured image,information (anchor information) indicating how many anchors exist inthe captured image and where each anchor exists within the capturedimage is obtained. The obtained anchor information is stored in the RAM212.

At step 1503, based on the anchor found at step 1502, the number of unitblocks existing within the captured image and the positions thereof arespecified. Specifically, by calculating the coordinates of the unitblock by referring to the above-described anchor information,information (unit block information) indicating the total number of unitblocks existing within the captured image and where each unit blockexists within the captured image is obtained. The obtained unit blockinformation is stored in the RAM 212.

At step 1504, a unit block on which attention is focused is determinedfrom the unit blocks existing in the captured image. At step 1505 thatfollows, extraction of additional information described previously isperformed by taking the unit block of interest as a target. Then, atstep 1506, whether or not extraction of embedded additional informationhas succeeded is determined. For example, in the case of FIG. 11Adescribed previously, on a condition that all pieces of additionalinformation corresponding to one unit block have been extracted, it isdetermined that the extraction has succeeded. Further, in the case ofFIG. 11B, on a condition that all pieces of additional informationcorresponding to one set of two adjacent unit blocks have beenextracted, it is determined that the extraction has succeeded. As thecase where a plurality of unit blocks is handled as one set, forexample, there may be a case where the first block is arranged on thefirst printed matter and the second block is arranged on the secondprinted matter. In such a case, a user captures the second printedmatter after capturing the first printed matter. Here, explanation isgiven by taking the case where the number of blocks is two and thenumber of pieces of printed matter is two as an example, but it is notnecessary to limit the case to this. In the case where the extraction ofadditional information has succeeded, this processing is terminated. Onthe other hand, in the case where the extraction of additionalinformation has failed, the processing advances to step 1507.

At step 1507, whether the extraction processing of additionalinformation has been performed for all the unit blocks existing withinthe captured image is determined. In the case where there is anunprocessed unit block, the processing returns to step 1504, and thenext unit block of interest is determined and the processing iscontinued. On the other hand, in the case where the extractionprocessing of additional information has been completed for all the unitblocks, the processing advances to step 1508.

At step 1508, the image capturing condition is changed for the nextimage capturing. For example, as described previously, the focusposition is changed automatically. The image capturing condition to bechanged is not limited to the focus position and for example, it mayalso be possible to change the zoom magnification at the same time sothat the image capturing target is captured larger. In addition, it mayalso be possible to change the exposure, the white balance, the imagequality mode of a built-in camera (for example, high-definition mode andlow-definition mode), the shutter speed, and so on. At this time, it issufficient to determine in advance the way which item is changed as atarget (amount of change and the like) in association with, for example,the format of image capturing-target printed matter, the image capturingenvironment, and so on. Further, it may also be possible to givepriority to each item and to change in the order from the item with thehighest priority. After the image capturing condition is changed, theprocessing returns to step 1501 and image capturing in accordance withthe newly set image capturing condition is performed and the processingis continued for a reacquired captured image as a target. Further, as amethod of changing the image capturing condition, it may also bepossible to change the image capturing condition in the case where theimage reading application determines No at step 1507 or another methodmay be used. For example, a user holds the mobile terminal in his/herhand and performs image capturing by bringing the position α into focus.Here, it is assumed that the data portion of the area 1202 in FIG. 12Bcannot be acquired from the image obtained by image capturing with theposition α being in focus. As a result of this, a progress barindicating progress of the extraction processing of additionalinformation, which is displayed by the image reading application, stopson the way. Then, there is a case where a user recognizes that theprogress bar has stopped on the way and changes the distance between themobile terminal 110 and the printed matter. Then, the image capturingcondition is changed in response to the change in the distance betweenthe mobile terminal 110 and the printed matter, and therefore, it ispossible to make use of the change of the image capturing condition asthe processing at step 1508. That is, the image capturing condition maybe changed without the image reading application notifying the built-incamera of the change of the image capturing condition.

The above is the contents of the image capturing control for acquiring acaptured image suitable to extraction of additional information. Due tothis, for example, in the example in FIG. 13 and FIG. 14 describedpreviously, in the case where image capturing is performed by bringingthe position α into focus in the first image capturing and extraction ofadditional information fails, image recapturing is performedautomatically by bringing the position β or γ into focus. Because ofthis, it is possible to efficiently obtain a captured image suitable toextraction of additional information. As the extraction processing ofadditional information, additional information may be extracted fromonly the image acquired by image recapturing, or additional informationextracted from each of a plurality of images obtained by a plurality oftimes of image capturing may be combined. Here, both pieces ofextraction processing are explained by taking an example. For example,it is assumed that image capturing is performed by bringing the positionα into focus and as a result of this, the data portions of the areas1201 and 1203 in FIG. 12A and FIG. 12B are extracted, but extraction ofthe data portion of the area 1202 fails. In such a case, it isdetermined that the extraction of additional information has failed atstep 1506. Then, it is possible to acquire the block at the position βby performing image recapturing by bringing the position β into focus.Here, additional information may be extracted from only the recapturedimage. Further, similarly as described above, for example, it is assumedthat image capturing is performed by bringing the position α into focusand as a result of this, the data portions of the areas 1201 and 1203 inFIG. 12A and FIG. 12B are extracted, but extraction of the data portionof the area 1202 fails. In such a case, it is determined that theextraction of additional information has failed at step 1506. Then, itis possible to acquire the block at the position β by performing imagerecapturing by bringing the position β into focus. At this time, thedata portions of the areas 1201 and 1203 within the block have alreadybeen acquired, and therefore, the CPU 210 makes an attempt to acquirethe data portion of the area 1202 of the block obtained by imagerecapturing. Here, in the case where the data portion of the area 1202of the block obtained by image recapturing is acquired, additionalinformation may be acquired by combining the data portions of the areas1201 and 1203 acquired by the first image capturing and the data portionof the area 1202 obtained by image recapturing. In this case, an attemptis made to acquire the data portion from only the area of the portionthat lacks at the time of image recapturing, and therefore, it ispossible to improve efficiency of the processing.

Further, it may also be possible to change the image capturing conditionother than the focus position, for example, the zoom magnification atthe same time. In the present embodiment, each time extraction ofadditional information fails, image recapturing is performed by changingthe image capturing condition, but this is not limited. For example, itmay also be possible to perform consecutive image capturing or imagecapturing of a moving image while changing the image capturing conditionand to sequentially perform the flow in FIG. 15 described above for aplurality of still images by consecutive image capturing or a pluralityof frame images making up a moving image as a target.

The additional information obtained by the above processing is displayedin parallel with the printed matter being captured. For example, amessage corresponding to the additional information is displayed bybeing superimposed on the printed matter being captured. Further, in thecase where the obtained additional information is a URL, a Web page or amoving image corresponding to the URL is displayed. In this case, thereis a possibility that an application corresponding to the URL is newlyactivated and only the page corresponding to the URL is displayed.

MODIFICATION EXAMPLE

The example is explained above in which extraction of additionalinformation is performed by using another captured image obtained byperforming image capturing by changing the focus position and the likein the case where additional information cannot be extractedsuccessfully. Next, as a modification example, an aspect is explained inwhich a combined image is generated from a plurality of captured imageswhose focus positions and the like are different and extraction ofadditional information is performed from the combined image.

FIG. 16 is a flowchart showing a flow of image capturing control foracquiring a captured image suitable to extraction of additionalinformation according to the present modification example. In thefollowing, explanation is given along the flow in FIG. 16, but thecontents in common to those of the flow in FIG. 15 described previouslyare omitted or simplified and in the following, different points areexplained mainly.

At step 1601, a plurality of captured images covering the entire printedmatter is acquired by image capturing in response to image capturinginstructions of a user via the UI of the mobile terminal 110. At thistime, as described previously, the image capturing condition, forexample, such as the focus position, is changed appropriately and thenimage capturing is performed. Data of a plurality of acquired capturedimages is stored in an area secured within the RAM 212.

At step 1602, a combined image corresponding to the entire printedmatter is generated by using a plurality of acquired captured images.Specifically, a combined image corresponding to the entire printedmatter is generated by modifying a plurality of captured images capturedso as to cover the printed matter into captured images with nodistortion by performing geometric transformation, for example, such asaffine transformation, for each of the plurality of captured images, andlinking the images after transformation together.

Step 1603 to step 1609 that follow correspond to step 1502 to step 1508,respectively, of the flow in FIG. 15. That is, first, the anchor searchdescribed previously is performed by taking the generated combined imageas a target (step 1603) and based on the found anchor, the number ofunit blocks within the combined image and the positions thereof arespecified (step 1604). Then, extraction processing of additionalinformation is performed for each unit block (steps 1605 and 1606) andin the case where additional information cannot be extracted from thecombined image, image capturing is performed again under a new imagecapturing condition (steps 1607 to 1609, step 1601). Then, theprocessing is continued by taking a new combined image generated basedon the recaptured image (step 1602) as a target.

The above is the contents of the image capturing control for acquiring acaptured image suitable to extraction of additional informationaccording to the present modification example. This modification exampleis effective in the case where the amount of additional information islarge, for example, in the case where the number of unit blockscorresponding to one set corresponding to one piece of additionalinformation is large, or the unit block itself, which serves as a base,has a large size. Further, in the case of additional information that ismeaningless unless each piece of additional information embedded in aplurality of pieces of printed matter is read, it may also be possibleto acquire a captured image (or combined image) corresponding to each ofthe plurality of pieces of printed matter and to generate an image bycombining the captured images.

As above, according to the present invention, in the case where imagecapturing for extracting additional information embedded in printedmatter is performed by using a built-in camera of a mobile terminal, aplurality of captured images whose image capturing condition is changedis used. Due to this, even in the case of unstable handheld imagecapturing, it is possible to obtain a captured image capable of readingof additional information and to easily extract additional information.

It may also be possible for the above-described image readingapplication to include a function to generate additional information, afunction to generate print data in which additional information isembedded, and so on, in addition to the function relating to extractionof additional information embedded in printed matter. That is, the imagereading application may include the functions of the above-describedembedment processing unit 101. By installing the image readingapplication such as this in a mobile terminal, it is also possible for auser to cause an image forming apparatus to print printed matter inwhich additional information is embedded and to extract additionalinformation from image capturing results of printed matter.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

According to the present invention, it is possible to easily extractadditional information from printed matter by using a built-in camera ofa mobile terminal.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2017-126735 filed Jun. 28, 2017, which is hereby incorporated byreference wherein in its entirety.

What is claimed is:
 1. An information processing apparatus that readsadditional information by capturing printed matter in which theadditional information is embedded by using a built-in camera, theinformation processing apparatus comprising: a control unit configuredto acquire a first captured image obtained by capturing the printedmatter under a first image capturing condition and a second capturedimage obtained by capturing the printed matter under a second imagecapturing condition different from the first image capturing condition;and an extraction unit configured to extract the additional informationby analyzing the first captured image and the second captured image. 2.The information processing apparatus according to claim 1, wherein in acase where extraction of the additional information from the firstcaptured image has succeeded in the extraction unit, the control unitdoes not acquire the second captured image and in a case whereextraction of the additional information from the first captured imagehas not succeeded in the extraction unit, the control unit reacquiresthe second captured image by changing the first image capturingcondition to the second image capturing condition and the extractionunit extracts the additional information by analyzing the first capturedimage and the second captured image.
 3. The information processingapparatus according to claim 1, wherein the additional informationincludes a plurality of data portions, and the extraction unit extracts,in a case where a first data portion arranged at a first position withina predetermined block of the first captured image is acquired and asecond data portion arranged at a second position within thepredetermined block is not acquired, the additional information byacquiring the second data portion arranged at the second position withinthe predetermined block of the second captured image and combining thefirst data portion acquired from the first captured image and the seconddata portion acquired from the second captured image.
 4. The informationprocessing apparatus according to claim 1, wherein the control unitacquires a captured image of the printed matter by performingconsecutive image capturing or image capturing of a moving image whilechanging the image capturing condition, and the extraction unit extractsthe additional information by analyzing a plurality of still imagesobtained by consecutive image capturing or a plurality of frame imagesmaking up a moving image.
 5. The information processing apparatusaccording to claim 4, wherein in a case where extraction of theadditional information has not succeeded in the extraction unit, thecontrol unit reacquires a captured image of the printed matter byperforming the consecutive image capturing or image capturing of amoving image again, and the extraction unit extracts the additionalinformation by analyzing a plurality of reacquired still images orreacquired frame images.
 6. The information processing apparatusaccording to claim 1, wherein the control unit generates a combinedimage from captured images in accordance with at least two kinds ofimage capturing condition, and the extraction unit extracts theadditional information by analyzing the combined image.
 7. Theinformation processing apparatus according to claim 6, wherein thecontrol unit generates, in a case where extraction of the additionalinformation has not succeeded in the extraction unit, the combined imageby reacquiring captured images in accordance with the at least two kindsof image capturing condition, and the extraction unit extracts theadditional information by analyzing the combined image generated basedon the reacquired captured images.
 8. The information processingapparatus according to claim 1, wherein the additional information isembedded repeatedly in units of blocks in image information on theprinted matter by being multiplexed, and the extraction unit extractsthe additional information from the captured image by performing ananalysis in the units of blocks.
 9. The information processing apparatusaccording to claim 1, wherein the image capturing condition is at leastone of a focus position, a zoom magnification, exposure, white balance,an image quality mode of the built-in camera, and a shutter speed.
 10. Acontrol method of a mobile information processing apparatus that readsadditional information by capturing printed matter in which theadditional information is embedded by using a built-in camera, thecontrol method comprising the steps of: acquiring a first captured imageobtained by capturing the printed matter under a first image capturingcondition and a second captured image obtained by capturing the printedmatter under a second image capturing condition different from the firstimage capturing condition; and extracting the additional information byanalyzing the first captured image and the second captured image. 11.The control method according to claim 10, wherein in a case whereextraction of the additional information from the first captured imagehas succeeded, the second captured image is not acquired, and in a casewhere extraction of the additional information from the first capturedimage has not succeeded, the second captured image is reacquired bychanging the first image capturing condition to the second imagecapturing condition and the additional information is extracted byanalyzing the first captured image and the second captured image. 12.The control method according to claim 10, wherein the additionalinformation includes a plurality of data portions, and in a case where afirst data portion arranged at a first position within a predeterminedblock of the first captured image is acquired and a second data portionarranged at a second position within the predetermined block is notacquired, the additional information is extracted by acquiring thesecond data portion arranged at the second position within thepredetermined block of the second captured image and combining the firstdata portion acquired from the first captured image and the second dataportion acquired from the second captured image.
 13. The control methodaccording to claim 10, wherein a captured image of the printed matter isacquired by performing consecutive image capturing or image capturing ofa moving image while changing the image capturing condition, and at theextraction step, the additional information is extracted by analyzing aplurality of still images obtained by consecutive image capturing or aplurality of frame images making up a moving image.
 14. The controlmethod according to claim 13, wherein in a case where extraction of theadditional information has not succeeded, a captured image of theprinted matter is reacquired by performing the consecutive imagecapturing or image capturing of a moving image again, and at theextraction step, the extraction is performed again by analyzing aplurality of reacquired still images or a plurality of reacquired frameimages.
 15. The control method according to claim 10, wherein a combinedimage is generated from captured images in accordance with at least twokinds of image capturing condition, and at the extraction step, theextraction is performed by analyzing the combined image.
 16. The controlmethod according to claim 15, wherein in a case where extraction of theadditional information has not succeeded, the combined image isgenerated by changing the image capturing conditions and reacquiringcaptured images in accordance with the at least two kinds of imagecapturing condition, and the extraction is performed by analyzing thecombined image generated based on the reacquired captured images. 17.The control method according to claim 10, wherein the image capturingcondition is at least one of a focus position, a zoom magnification,exposure, white balance, an image quality mode of the built-in camera,and a shutter speed.
 18. A non-transitory computer readable storagemedium storing a program for causing a computer to perform a controlmethod of a mobile information processing apparatus that readsadditional information by capturing printed matter in which theadditional information is embedded by using a built-in camera, thecontrol method comprising the steps of: acquiring a first captured imageobtained by capturing the printed matter under a first image capturingcondition and a second captured image obtained by capturing the printedmatter under a second image capturing condition different from the firstimage capturing condition; and extracting the additional information byanalyzing the first captured image and the second captured image.